Most types of computer systems have a requirement to maintain data for prolonged periods of time. To meet this requirement, a typical computer system includes a coupling to a data storage system which the computer system can access to store and retrieve the data. The computer system may be coupled to the data storage system via a high speed data transfer interface (e.g., a small computer system interface (SCSI), a Fibre-Channel interface (e.g., ESCON), or the like), or the coupling may be formed over a computer network such as a Storage Area Network (SAN) that may link a plurality of computer systems to one or more high-capacity data storage systems. Through an interoperation of software (e.g., applications, operating systems, protocols and the like) and hardware (e.g., circuitry) in both the computer system and the data storage system, the computer system is able to access data within storage media (e.g., disk drives) that the data storage system controls.
Within a typical data storage system, one or more processors (e.g., Central Processing Units or CPUs) operate according to prescribed software program(s) to manage and control access to the data within the storage media in the data storage system on behalf of the computer systems that request access to such data. Such data storage system software programs are generally considered the operating system or control program for the data storage system. For example, within a high-capacity data storage system such as one of the Symmetrix line of data storage systems manufactured by EMC Corporation of Hopkinton, Mass., U.S.A., a front end interface provides a coupling for the data storage system to one or more computer system(s) (via direct interfaces or via a SAN) while a back end interface provides a coupling to the storage media devices (e.g., disk drives) within the data storage system that stores data. The front and back end interfaces are coupled by a data bus (one or more) which allows the interfaces to interoperate with each other. A cache memory system is accessible on the data bus for use by the front and back end interfaces to temporarily store data during processing. A processor operating within the front end interface (e.g., on a circuit board that operates as the front end interface) operates a software program (e.g., firmware or microcode) that performs high speed processing of data (and requests for such data) between the front end interface and the remotely connected computer systems. Likewise, the back end interface includes a processor that operates a software program to handle tasks associated with accessing (e.g., reading and writing) data to and from the storage devices within the data storage system based on the requests received by the front end interface.
Due to the complex operation of a typical data storage system, the software programs which perform (e.g., execute or otherwise operate) on processor(s) such as the front and back end interfaces within a data storage system can become quite large and complex. By way of example, the microcode software program which provides the access request processing operations for a front end interface within a Symmetrix data storage system may be many thousands of lines of code in length. During the design, development and testing of such complex data storage system software control programs, software developers frequently include the ability for the software program to operate in a “trace mode” which allows the program to trace the occurrence of certain trace events during the program's operation. Essentially, trace mode operation causes the software program to capture trace data as defined by a software developer in relation to an occurrence of the certain defined or selected trace events.
Before operating a software program in trace mode in a conventional data storage system, a software developer is able to define one or more trace events and associated trace data which is to be captured upon occurrence of each trace event. The software developer can then operate the software program in trace mode. While operating in trace mode, the software program in the data storage system is able to detect occurrences of each trace event during operation of the software program. Upon detection of a trace event, the software program performs or calls a designated trace routine (which is itself generally considered part of the software control program) which is responsible for capturing trace data (e.g., variable or data structure values, data access request formats, command parameters, and so forth) related to the trace event. The trace routine places the trace data, which may include the current values of data structures, parameters, input/output request values, and so forth that are relevant to the trace event as designated by the software developer, within a trace buffer in the cache memory system within the data storage system. The trace buffer is typically a reserved area of the cache memory system which is limited in size, for example, to sixteen or thirty-two megabytes (MB).
After the software developer operates the software program in trace mode for a certain period of time in order to exercise the features of the program which would typically cause the trace event(s) to occur, the software developer can halt the operation of the software program. At this point, the trace buffer in the cache memory system contains the trace data which the trace routines captured during operation of the software program at the occurrence of each trace event. The software developer can then view the trace data within the trace buffer in the cache memory system by interaction, for instance, with a service processor (e.g., a keyboard and monitor) which is integrated as part of the data storage system. By reviewing the trace data, the software developer can determine if the software program for which the trace data was generated had properly performed in the data storage system during its operation.
Some data storage system configurations include a service processor that allows the software developer to download the trace data from the trace buffer onto a removable storage medium such as a floppy disk so that the trace data can be transported to another computer system for further analysis. By viewing the trace data according to these techniques, the software developer can debug the software program to determine whether or not it is operating properly.